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I. 


PROGRESS  MADE  DURING  PERIOD,  PROBLEMS,  AND 
RECOMMENDATIONS 


Progress 

Little  progress  was  made  on  the  Phase  I  effort  in  the  past  two  months.  We  are  currently  awaiting 
the  test  data  set,  which  is  expected  in  the  very  near  future.  Figure  1  shows  the  design  for 
processing  the  test  data  set.  The  approach  is  to  use  the  hierarchical  neural  network  described  in 
the  previous  progress  report,  to  fuse  not  only  a  variety  of  features  extracted  from  a  single  channel 
of  data,  but  also  to  fuse  information  across  sensors.  As  with  previously  described  processing 
results,  not  all  of  the  data  paths  indicated  are  likely  to  be  required.  The  data  and  first-level 
network  processing  results  will  indicate  which  features  to  keep  for  good  statistical  processing 
performance. 

The  results  from  the  first  two  months  were  used  to  write  a  conference  paper  (enclosed  as 
Appendix  A). 

Specific  progress  relative  to  the  SOW  follows: 

1.  Identify  Features  and  Parameters  to  Use 

All  feature  extractors  proposed  have  been  developed  and  coded  onto  the  ORINCON  computer 
system.  Parameters  for  processing  will  be  identified  when  the  data  of  interest  is  received. 

2.  Develop  Feature  Set  Processing 

Completed.  Additional  features  will  be  included  as  they  are  developed  on  other  ORINCON 
projects. 

3.  Identify  Data  Sets  /  Segments  from  Westland  Helicopters 

This  task  has  been  replaced.  The  government  will  supply  the  data  and  ORINCON  will  process 
an  additional  fault  data  set. 


4.  Transfer  and  Format  Data  for  In-House  Use 

Completed  for  the  Hollins  data  set,  but  still  awaiting  the  test  data  set. 

5.  Process  Real  Data  Through  the  Feature  Extractors  and  Assess  Utility 


Not  started  for  the  test  data  set. 

6.  Develop  Training  Data  Set 

Not  started  for  the  test  data  set. 


7.  Train  Neural  Net 

Not  started  for  the  test  data  set. 

8.  Develop  Test  Data  Set 
Not  started  for  the  test  data  set. 
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9. 


Test  the  System 

In  progress  for  the  HoUins  data.  Development  of  specific  performance  measures  is  in  progress. 
We  are  currently  modifying  the  processing  to  include  multichannel  inputs  as  shown  in  Figure  1. 

10.  Demonstrate  the  Prototype  System 
Not  started  yet. 

11.  Assess  System  Performance  and  Produce  a  Final  Report 
Not  started  yet. 

12.  Process  an  Additional  Fault  Detection  Data  Set 

Not  started  yet.  Awaiting  the  additional  data  set. 

Problems 

We  are  currently  waiting  for  the  government  data  that  is  of  interest.  Most  of  the  development 
has  been  done  using  a  system  developed  for  an  IR&D  project  and  based  on  the  ORINCON 
PRISM  system.  The  system  allows  very  fast  training  and  prototype  system  development  in 
trying  out  different  signal  processing  parameters  and  feature  extractors.  Unfortunately,  the 
system  is  incompatible  with  the  KHOROS  software  recommended  by  the  government.  The 
outputs  from  the  system  can  be  made  compatible  with  KHOROS  for  playback.  However,  some 
time  would  be  required  to  change  the  software  modules  to  the  KHOROS  format. 

The  current  government- supplied  data  set  was  originally  flagged  in  the  contract  as  “the 
additional  data  set”  (see  12  above).  Thus  if  all  the  contractors  are  to  receive  this  data  set,  we 
need  to  identify  an  additional  data  set. 

Recommendations 

None. 

n.  RESULTS  RELATED  TO  PREVIOUSLY  IDENTIFIED  PROBLEM  AREAS 

The  fire  pump  data  has  been  identified  as  the  test  data  set  to  be  processed  on  the  contract.  That 
data  set  is  expected  to  be  in-house  shortly. 

ni.  DELIVERABLES  COMPLETED  AND  DELIVERED 

Completed  CDRLs  Date  Delivered 

1.  000 lAB  Progress  Report  August  31  1992  (this  deliverable) 

rv.  SUBCONTRACTING  AND  RESULTS  DURING  REPORTING  PERIOD 
None  to  report  at  this  time. 

V.  EXTENT  OF  TRAVEL 

Not  applicable. 
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ABSTRACT 

A  problem  of  current  interest  is  the  automatic  detection  and 
classification  of  faults  in  mechanical  systems  such  as  the 
gearboxes  and  transmissions  on  board  helicopters.  Current  fault 
processing  uses  relatively  simple  metrics  to  characterize  changes 
in  measured  vibration  data.  The  metric  speciHcation  and 
thresholds  for  detection  and  classification  are  found  using 
complicated  analytic  models  for  the  gearboxes.  With  this 
approach  fmding  a  solution  to  the  problem  is  difficult  since  the 
understanding  of  the  problem,  the  metrics  that  can  be  used,  and 
the  fault  classes  that  are  accounted  for  are  only  as  good  as  the 
model  developed.  In  many  cases,  the  interaction  of  fault 
conditions  with  the  mechanical  system  is  time  varying  and 
highly  nonlinear. 

An  alternative  solution  described  here  is  to  use  generalized 
time -frequency  and  time-scale  representations  coupled  with  a 
hierarchy  of  neural  nets.  The  processing  assumes  no  imderlying 
model  for  tliw  events  of  interest.  Rather  the  system  'learns'  to 
detect  and  classify  faults  by  examination  and  fuion  of  features 
from  training  data  which  have  known  fault  conditions.  Results 
of  processing  real  helicopter  gearbox  vibration  data  with  seeded 
faults  are  shown. 


1.  INTRODUCTION 

Described  here  is  the  application  of  time-frequency  and 
time-scale  representations  (TFRs  and  TSRs)  for  the 
characterization  of  vibration  data.  The  TFRs  and  TSRs  form 
inputs  to  a  hierarchy  of  neural  nets  to  solve  for  the  detection  and 
classification  of  faults  in  helicopter  gear  boxes  and 
uansmissions.  The  approach  is  general  and  can  be  applied  to  a 
variety  of  detection  /  classiScation  problems.  Figure  1  shows  a 
functional  flow  diagram  for  the  system  considered  here.  The 
first  row  of  boxes  p^orm  feature  extraction  or  characterization 
of  the  input  data.  Feature  extraction  estimates  a  vector  time 
series  of  features  from  the  input  data  for  a  selected  time 
segment.  The  feature  vector  forms  the  input  retina  for  the 
multiple  neural  nets.  The  neural  nets  perfnm  a  mapping  from 
the  input  retina  -  feature  space  to  an  output  class.  The 
hierarchical  neural  network  allows  for  the  fusion  of  a  variety  of 
features  [1],  The  approach  can  be  expanded  to  include 
additional  features  as  well  as  multiple  sensors,  alternative 
sensors,  and  additional  classifiers.  Each  of  the  neural  networks 
in  the  system  has  outputs  that  correspond  to  normal  operation  as 
well  as  a  specific  fault  conJidon.  The  inputs  to  the  First  layer 
nets  are  the  single  feature  extractors,  while  the  inputs  to  the 
second  layer  net  are  the  outputs  from  the  first  layer  nets. 


Input 

Data 


m 

Wavelets 

AOK 

TFR 

ftony 

Spectra 

1 

1 

♦ 

iKisziai 

3  Layer 

3Uyer 

Perceptron 

Perceptron 

Neurd  Net 

Neural  Net 

Merge 

Activations 


Output  Activations  for 

AUCUaies 


Stayer 
Percepttoo 
Neural  Net 


Output  Oats 


Figure  1 TFR  /  TSR  -  Hierarchical  Neural  Net 

A  problem  in  the  design  of  the  system  is  the  selection  of 
appropriate  features.  The  goal  in  selecting  features  is  to  choose 
a  set  ^at  can  characterize  signals  of  interest  sufficiently  so  that 
those  classes  are  well  separated  when  solving  the  classification 
problem.  We  believe  that  the  best  "features"  to  use  to 
characterize  signals  are  not  the  standard  scalar  features  that 
commonly  are  considered  (such  as  duration,  center  frequency 
and  bandwidth),  but  rather  a  compressed  version  of  the  input 
signal  as  represented  the  "features"  found  with  the  TFRs  and 
TSRs.  Four  techniques  are  considered  here.  They  are  the 
standard  short  time  Fourier  transform  (S  in  ;,  the  radially  - 
Gaussian  adaptive  optimal  kernel  time-fiequency  representation 
(AOK  TFR)  [2],[3],  Prony's  method  [4],  and  the  wavelet 
transform  using  the  Gabor  wavelet  [S]. 

Details  of  the  feature  extractors  and  the  hierarchical  neural 
net  are  given  in  section  2.  The  application  of  the  system  for 
fault  detection  /  classification  of  teal  helicopter  geartox  data  is 
presented  in  section  3.  The  architecture  shown  in  figure  1 
allows  for  the  analysis  of  the  utility  of  each  feature  extractor. 
For  the  data  set  considered  here  the  analysis  indicates  that  the 
STFT  and  Wavelet  transforms  are  sufficient  features  to  resolve 
the  differeiu  fault  conditions.  Section  4  contains  conclusions . 
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2.  THE  PROCESSING 


Details  of  the  different  system  components  are  described 

here. 

2.1  The  STFT 

The  short-time  fast  Fourier  transform  (STFT)  feature  is 
found  as  follows:  let  x(t)  represent  the  input  data  set;  let  X^CO 
represent  the  FFT  of  the  n-th  segment  of  data.  That  is, 

1  .\-l  -j— 

N  ,=o  (1) 


where  typically  T  =  N/2  (i.e.,  “50%  overlap")  and  N  is  a  power 
of  2.  w(t)  is  a  window  function,  such  as  the  Hamming  window. 
The  feature  is  a  set  of  FFT  coefficients.  The  set  forms  the 
feature  space  for  any  particular  time/fiequency  segment  of  the 
input  data; 


{log(|X„(f)|),t,<n<t2,f,<f<f2} 


2,2  The  AOK  TFR 

An  instantaneous  time  /  frequency  representation  (TFR) 
gives  a  high  resolution  characterization  of  the  data  in  time  as 
well  as  FFT  resolutions  in  frequency  for  signals  of  interest.  The 
particular  TFR  that  we  use  here  is  the  adaptive  optimal  radially- 
Gaussian  kernel  TFR  developed  by  Baraniuk  and  Jones  [2].  The 
TFR  uses  a  radially-Gaussian  signal-dependent  kernel  that 
changes  shape  to  optimally  smooth  the  distribution. 

The  optimal  kernel,  4>,  for  a  signal  is  defined  as  the 
solution  to  the  following  optimization  problem: 

<l>(r,v)|^rdrdv 

subject  to 


<t>(r,\|/)  =  e 


(3) 


(4) 


v)j^rdrdv ^ a,  a ^ 0 

.  (5) 

A(r,4')  is  the  ambiguity  function  (AF)  of  the  signal  in  polar 
coordinates.  Once  the  optimal  kernel  is  computed,  the  TFR  is 
given  by 

P(t,(D)  =  A(0.x)  <I>(0.T)  e-^®‘-j'‘"d0dx 

2ji  (6) 

The  representation  is  good  for  characterizing  short  duration 
and  nonstationary  events.  The  AOK  TFR  is  computationally 
expensive.  However,  in  the  fault  detection  application,  it  is  not 
required  to  run  in  real  time.  The  system  is  oivly  required  to 
output  detection  /  classification  results  at  a  relatively  low  rate. 
As  with  the  STFT  feature  extractor,  a  time  sequence  of  the  AOK 
TFR;  form  the  input  retina. 


2J  Prony's  Model  Method 

Prony's  model  method  assumes  the  signals  of  interest  are 
modeled  by  a  sum  of  damped  sinusoids.  The  model  is  well 
suited  for  characterizing  impulsive  type  of  events.  The  resulting 
model  gives  a  variety  of  parameters  that  may  be  exploited  in 
characterizing  vibration  waveforms.  The  Prony  model  is  of  the 
form : 

x[n]  =  ^  Afc  expf(ait  )(”  -  1)T  +  j^k  1 

k=l  (7) 

where  zfn]  is  the  observed  time  series  data,  p  is  the  model  order, 
Ak  is  the  amplitude  of  the  k-th  coefficient,  is  the 
corresponding  damping  term,  ^  is  the  center  frequency,  T  is  the 
sample  interval,  and  djk  t*  initial  phase.  The  parameters  of 
the  model  can  be  estimated  using  least  squares  techniques  [4]. 
Several  different  noise  discrimination  techniques  have  been 
developed  for  use  with  the  Prony  model  method  [6].  In  the 
processing  presented  here  a  time  sequence  of  spectral  estimates 
derived  from  Prony's  model  fit  to  the  data  is  us^ 

2.4  Wavelet  Processing 

The  wavelet  transform  (WT)  is  a  time  domain 
representation  of  a  signal  in  terms  of  dilated  and  shifted  versions 
of  suitable  analyzing  wavelets.  Here  we  use  a  discrete 
approximation  of  the  continuous  wavelet  transform.  The 
analyzing  wavelet  used  is  the  analytic  portion  of  the  Gabor 
function; 

h(t)  =  sin(a)ot) 

A  time  sequence  of  wavelet  transforms  forms  the  input 
feature  space  to  the  hierarchical  neural  net  classifier. 

2.5  Hierarchical  Neural  Net  Processing 

The  feature  sets  are  input  to  the  hierarchical  neural  net  for 
detection  and  classification  processing.  The  hierarchical  net 
shown  in  figure  1  has  been  successfully  used  to  fuse  multiple 
features  together  so  as  to  increase  the  probability  of  detection 
and  correct  classification  while  at  the  same  time  reducing  the 
probability  of  false  alaim  [1]  Here  each  neural  net  is  a  three 
layer  perceptron.  The  first  layer  of  nets  process  the  single 
feature  vectors  .  The  output  from  the  first  layer  of  nets  is  then 
input  to  a  second  layer  to  'fuse'  the  results  from  the  first  layer 
together.  For  the  hierarchical  multi-network  architecture, 
single-feature  exemplars  are  selected  and  used  to  train  the  set  of 
four  single-feature  neural  networks  indqjendendy.  To  finish  the 
training  of  the  hierarchical  multi-network  architecture,  each  of 
the  trained  single  feature  neural  nets  are  run  on  the  training 
data.  The  various  outputs  from  the  single  feature  neural 
networks  are  aggregated  together  to  form  the  input  to  the 
hierarchical  neural  network  as  shown  in  Figure  1.  A  second 
exemplar  selection  is  performed  for  training  of  the  second  layer 
net 
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3.  REAL  DATA  PROCESSING  RESULTS 

The  data  set  used  was  collected  by  Mark  Hollins  of  the 
Naval  Air  Test  Center.  It  consists  of  vibration  data  from  the  tail 
rotor  transmission  of  a  TH-IL  helicopter.  Data  for  normal  and 
five  induced  faults  conditions  were  collected.  Initially  the  four 


feature  extrastors  described  were  considered.  Table  I  shows  the 
parameter  settings  used.  For  the  results  described  a  detailed 
investigation  of  the  feature  extraction  parameters  was  not  made. 
The  only  parameter  adjusted  was  the  time  duration  (or  extent)  of 
the  input  retina  to  the  fusion  network  of  the  hierarchical 
network. 

Each  of  the  neural  networks  used  is  a  three  layer 
perceptron.  Training  is  performed  until  the  RMS  training  error 
from  each  of  the  networks  is  approximately  0.01.  In  past  work 
at  ORINCON  this  RMS  error  has  resulted  in  networks  that  give 
robust  detection  /  classiHcation  performance  for  both  training 
and  test  data  sets.  Visual  inspection  of  the  single  feature 
network  outputs  on  test  data  resulted  in  the  AOK  TFR  and 
Prony  model  spectra  being  dropped  from  the  fusion  network 
processing  (i.e.  the  dotted  lines  in  figure  1  show  connections  that 
were  initially  considered  but  are  not  included  in  the  final  system 
described  here).  It  is  important  to  note  that  these  results  only 
apply  to  the  Hollins  data  set  and  may  not  hold  for  additional  data 
sets. 

Figure  2  show  specific  results  for  the  ’bearing  outer  race 
fault'  class  category  for  the  Hollins  data  set.  There  are  five 
separate  windows  in  the  display  shown  in  the  figure.  'The 
windows  labeled  ’fft’,  'prony',  'AOK',  and  'WAVELET 
correspond  to  individual  feature  extractors  as  well  as  the  output 
activations  from  the  single  feature  networks  (see  the  'Prony' 
window  in  figure  2).  The  network  activations  correspond  to: 
Column  5  =  Normal  gearbox. 

Column  0  =  Bearing  inner  race  fault. 

Column  1  =  Bearing  rolling  element  fault. 

Column  2  =  Bearing  outer  race  fault. 

Column  3  =  Gear  spall  fault  and. 

Column  4  =  Gear  1/2  tooth  cut  fault. 

The  neural  net  outputs  are  intensity  encoded  (the  real 
system  uses  color  /  temperature  encoding).  Dark  values  indicate 
a  relatively  low  activation  level.  White  indicate  a  relatively  high 
activation  level.  The  feature  extractors  show  the  same  amount 
(or  time  duration)  of  data,  however  each  extractor  uses  different 
processing  parameters  so  that  the  output  scan  rates  are  different. 
For  example  for  the  'prony'  and  WAVELET  feature  extractors 
there  are  four  output  scans  for  each  output  scan  of  the  'fft' 
extractor.  The  'AOK'  extractor  outputs  eight  scans  for  each  of 
the  'fft'  extractor. 

The  window  labeled  'fiision.net'  shows  the  inputs  and  the 
output  for  the  second  layer  net  in  the  hierarchical  neural  net 
The  input  features  are  the  outputs  from  the  FFT  and  Wavelet 
first  layer  networks  as  shown  in  figure  1.  The  FFT  output  from 
the  first  layer  have  been  stretched  in  time  so  as  to  synchronize 
with  the  faster  running  Wavelet  feature  extractor.  The  figure 
shows  results  for  processing  of  test  data  that  contains  only  the 
event  of  interest. 

Each  of  the  extractors  gives  a  different  spectral 
representation  of  the  data.  The  ideal  neural  net  performance 
would  result  in  the  column  corresponding  to  the  gearbox 
condition  being  bright  white  when  data  for  that  condition  is 
input  to  the  system  and  black  for  the  other  fault  conditions.  For 
example  in  figure  2  the  third  column  over  fiebeled  column  2) 
should  be  bright  white  while  all  the  other  coluiiuts  should  be 
dark  for  perfect  processing.  Drop  outs  in  the  ’Normal  geaibox' 
class  column  or  activations  in  other  columns  correspond  to  false 
alarms  with  in  the  system  when  no-fault  data  is  input.  Low  or 
no  activations  for  the  event  of  interest  column,  when  the  event  is 
present,  corresponds  to  missed  detections  or  missclassifrcations. 
As  seen,  for  all  of  the  single  feature  nets  there  are  significant 
dropouts  and  missclassifications.  The  Prony  and  AOK  TFR  nets 


give  the  poorest  performance.  The  FFT  and  Wavelet  nets 
performance  are  still  far  from  ideal.  However,  the  output  from 
the  fusion  net  is  close  to  ideal.  Figures  similar  to  that  shown  in 
2  for  both  normal  and  other  fault  conditions  give  equally  good 
results. 

The  fusion  net  uses  a  fairly  long  integration  time  (200  scans 
which  is  about  1/3  the  length  of  the  display  shown  in  the  fusion 
net).  Long  integration  times  (i.e.  retinas  with  long  time  extent) 
are  possible  with  the  second  net  in  the  hierarchical  network. 
This  is  because  there  are  relatively  few  inputs  to  the  network 
(here  there  are  only  12  inputs  to  the  second  layer  network).  The 
first  layer  nets  are  restricted  to  have  short  time  extent  (unless 
averaging  of  time  scans  is  performed).  This  is  because  a  scan  is 
typically  made  up  of  128  or  2S6  points  so  that  large  numbers  of 
scans  are  not  computationally  tractable. 

4.  CONCLUSIONS 

We  have  shown  that  the  use  of  multiple  TFRs  and  TSRs 
coupled  with  a  hierarchical  neural  net  approach  appears  to  work 
well  for  characterization,  detection,  and  classification  of  faults 
from  vibration  dau  in  mechanical  systems.  The  results  indicate 
that  only  the  FFT  and  Wavelet  feature  extractors  are  required  for 
detection  and  classification  in  the  Hollins  data  set.  The 
parameters  that  we  have  selected  result  in  single  feature  retinas 
that  cover  half  to  a  hill  rotation  of  the  gearbox  shaft.  Multiple 
revolutions  are  integrated  together  in  the  second  layer  of  the 
hierarchical  net  Further  investigation  of  the  parameters  used  in 
the  single  feature  extractors  is  required  to  develop  processing  to 
identify  alternative  fault  conditions. 

llie  approach  described  is  general  and  can  be  applied  to  a 
variety  of  problems;  it  has  has  been  applied  to  underwater 
acoustic  problems  and  diagnosis  of  blood  flow  from  ultra  sound 
measurements  with  equal  success.  The  hierarchical  neural  net 
can  also  be  used  as  a  tool  for  the  analysis  of  the  utility  of 
individual  feature  extractors  for  solution  of  the  detection  / 
classification  problem.  In  addition  to  the  problem  described 
here,  the  hierarchical  neural  net  can  be  us^  to  fuse  multiple 
sensors,  alternative  sensors,  as  well  as  differem  classifiers. 
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Figure  2.  Fault  Detection  Processing  Example  :  Bearing  Outer  Race  Fault 
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